Bidirectional American Sign Language to English Translation

In the US alone, there are approximately 900,000 hearingimpaired people whose primary mode of conversation is sign language. For these people, communication with non-signers is a daily struggle, and they are often disadvantaged when it comes to finding a job, accessing health care, etc. There are a few emerging technologies designed to translate sign language to English in real time, but most of the current research attempts to convert raw signs into English words. This aspect of the translation is certainly necessary, but it does not take into account the grammatical differences between signed languages and spoken languages. In this paper, we outline our bidirectional translation system that converts sentences from American Sign Language (ASL) to English, and vice versa. To perform machine translation between ASL and English, we utilize a generative approach. Specifically, we employ an adjustment to the IBM word-alignment model 1 (IBM WAM1)[14] where we define language models for English and ASL, as well as a translation model, and attempt to generate a translation that maximizes the posterior distribution defined by these models. Using these models, we are able to quantify the concepts of fluency and faithfulness of a translation between languages.